This is the html-rendered R markdown file (Rmd) which accompanies the manuscript:
Huber-Huber, C. & Melcher, D. (2020) The behavioural preview effect with faces is susceptible to statistical regularities: Evidence for predictive processing across the saccade. Manuscript submitted for publication.

The code used to generate this file reproduces all statistics and figures in the manuscript from the data which is provided as well on the OSF page of the project (https://osf.io/ty69k/). To regenerate this file from the raw data, set the variables IMPORTDATAANEW and RUNMODELLINGANEW in the code chunk called ANALYSIS SETTINGS to TRUE.

This html file also contains all supplementary figures and tables and the analysis of the training phase data which is not directly relevant to the study but shows some very interesting insights (credit to an anonymous reviewer). For all background information about the study, however, please refer to the manuscript file.

Figure and table numbers correspond to figures and tables in the manuscript.

1 Overview of the data

1.1 Proportion correct

Here, we check the average performance across the experiment in the tilt discrimination task. Participants with less than 60% correct responses are excluded assuming that they did not do the task.

    partnr Training prop.corr prop.corr<0.60
 1:     33  Invalid 0.5191364           TRUE
 2:     37  Invalid 0.5322266           TRUE
 3:      8  Invalid 0.5430528           TRUE
 4:     34    Valid 0.5475709           TRUE
 5:     27  Invalid 0.5616438           TRUE
 6:     40    Valid 0.5714286           TRUE
 7:     18  Invalid 0.6050831          FALSE
 8:     12  Invalid 0.6113744          FALSE
 9:     19    Valid 0.6174168          FALSE
10:     36    Valid 0.6656863          FALSE
11:     25  Invalid 0.7247796          FALSE
12:     38    Valid 0.7431641          FALSE
13:     29  Invalid 0.7446184          FALSE
14:     22  Invalid 0.7497556          FALSE
15:     16  Invalid 0.7526882          FALSE
16:     15    Valid 0.7563601          FALSE
17:     32  Invalid 0.7608696          FALSE
18:     11    Valid 0.7617647          FALSE
19:      5    Valid 0.7626953          FALSE
20:      3  Invalid 0.7689282          FALSE
21:     17    Valid 0.7763672          FALSE
22:      2    Valid 0.7778865          FALSE
23:      6  Invalid 0.7864838          FALSE
24:     26    Valid 0.8164062          FALSE
25:     39  Invalid 0.8189824          FALSE
26:      9    Valid 0.8258317          FALSE
27:     24  Invalid 0.8406647          FALSE
28:     10  Invalid 0.8437500          FALSE
29:      7    Valid 0.8447266          FALSE
30:     13    Valid 0.8476562          FALSE
31:     28    Valid 0.8554688          FALSE
32:     35  Invalid 0.8563050          FALSE
33:     20  Invalid 0.8631476          FALSE
34:     21    Valid 0.8875855          FALSE
35:     31  Invalid 0.8895406          FALSE
36:     23    Valid 0.8935547          FALSE
37:      4  Invalid 0.8955665          FALSE
38:     41  Invalid 0.8970588          FALSE
39:     14  Invalid 0.9111328          FALSE
40:     30    Valid 0.9149560          FALSE
41:      1    Valid 0.9274510          FALSE
    partnr Training prop.corr prop.corr<0.60

Thus, participants excluded are: 33, 37, 8, 34, 27, 40.

Number of participants per training group:

Training
  Valid Invalid 
     17      18

1.2 Demographics

Age:

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   18.0    20.0    21.0    22.5    24.0    41.0

Gender (0 -> female; 1 -> male):

gender
 0  1 
25 10

Handedness (0 -> left; 1 -> right):

handedness
 0  1 
 6 29

Eyedness (0 -> left; 1 -> right):

eyedness
 0  1 
14 21

2 Response time

For details on the model fitting and model comparison approach, see the corresponding Rmd source code file.

2.1 Number of trials

Trials in the RT analysis: 12671, i.e. 70.5826649% of a total number of 17952 trials.

2.2 Model comparisons

Supplementary Table S1. Response time model comparisons to determine the random effects structure. Each column presents the parameters (random and fixed) of a model. Models in adjacent columns are compared to each other by likelihood ratios tests. Test results (χ2, degrees of freedom, and p value) for a model pair are printed for the right model of each pair in the last three rows. Observations - the number of single trials for the model; AICc - Akaike’s Information Criterion corrected for small sample sizes; Df - model degrees of freedom; χ2 - statistic for the likelihood ratio test, for each model the difference in deviance compared to the model to the left; χ2 Df - degrees of freedom for the likelihood ratio test, for each model the difference in Df compared to the model to the left; p - the p-value for the likelihood ratio test.
Dependent variable:
-1 / Response time [sec]
Fitting method:
REML REML REML REML
(1) (2) (3) (4)
Random effects variances
Participant
(Intercept) 0.039 0.049 0.049 0.049
Target Orientation (In-Up) 0.002 0.002 0.002
Preview (Inv-Val) 0.001 0.001 0.001
Trial number 0.010 0.010 0.010
Target Orientation x Preview 0.003 0.003 0.003
Target Orientation x Trial number 0.0001 0.0001 0.0001
Preview x Trial number 0 0
Target Orientation x Preview x Trial number 0
Residual Variance 0.039 0.036 0.036 0.036
Fixed effects
Target Orientation (In-Up) 0.033 0.037 0.037 0.037
(0.007) (0.010) (0.010) (0.010)
t = 4.675 t = 3.749 t = 3.749 t = 3.749
Preview (Inv-Val) 0.044 0.041 0.041 0.041
(0.007) (0.009) (0.009) (0.009)
t = 6.143 t = 4.549 t = 4.549 t = 4.549
Training (Inv-Val) -0.144 -0.138 -0.138 -0.138
(0.068) (0.075) (0.075) (0.075)
t = -2.128 t = -1.831 t = -1.831 t = -1.831
Trial number -0.074 -0.077 -0.077 -0.077
(0.004) (0.017) (0.017) (0.017)
t = -20.736 t = -4.388 t = -4.388 t = -4.388
Target Orientation x Preview -0.010 -0.014 -0.014 -0.014
(0.014) (0.016) (0.016) (0.016)
t = -0.729 t = -0.841 t = -0.841 t = -0.841
Target Orientation x Training 0.010 0.002 0.002 0.002
(0.014) (0.020) (0.020) (0.020)
t = 0.696 t = 0.083 t = 0.083 t = 0.083
Preview x Training -0.074 -0.070 -0.070 -0.070
(0.014) (0.018) (0.018) (0.018)
t = -5.242 t = -3.848 t = -3.848 t = -3.848
Target Orientation x Trial number 0.016 0.013 0.013 0.013
(0.007) (0.007) (0.007) (0.007)
t = 2.269 t = 1.866 t = 1.866 t = 1.866
Preview x Trial number 0.001 0.003 0.003 0.003
(0.007) (0.007) (0.007) (0.007)
t = 0.079 t = 0.425 t = 0.425 t = 0.425
Training x Trial number 0.065 0.058 0.058 0.058
(0.007) (0.035) (0.035) (0.035)
t = 9.151 t = 1.676 t = 1.676 t = 1.676
Target Orientation x Preview x Training 0.015 0.010 0.010 0.010
(0.028) (0.033) (0.033) (0.033)
t = 0.522 t = 0.300 t = 0.300 t = 0.300
Target Orientation x Preview x Trial number 0.007 0.012 0.012 0.012
(0.014) (0.014) (0.014) (0.014)
t = 0.486 t = 0.854 t = 0.854 t = 0.854
Target Orientation x Training x Trial number -0.007 -0.002 -0.002 -0.002
(0.014) (0.014) (0.014) (0.014)
t = -0.469 t = -0.122 t = -0.122 t = -0.122
Preview x Training x Trial number 0.033 0.028 0.028 0.028
(0.014) (0.014) (0.014) (0.014)
t = 2.359 t = 2.051 t = 2.051 t = 2.051
Target Orientation x Preview x Training x Trial number -0.015 -0.017 -0.017 -0.017
(0.028) (0.027) (0.027) (0.027)
t = -0.541 t = -0.618 t = -0.618 t = -0.618
Grand mean -1.025 -1.022 -1.022 -1.022
(0.034) (0.038) (0.038) (0.038)
t = -30.344 t = -27.213 t = -27.213 t = -27.213
Observations 12,671 12,671 12,671 12,671
AICc -4852.342 -5702.931 -5700.924 -5698.916
Log Likelihood 2444.198 2874.509 2874.509 2874.509
Deviance -4888.396 -5749.019 -5749.019 -5749.019
Df 18 23 24 25
χ2 860.623 0 0
χ2 Df 5 1 1
p < .001 1.000 1.000
Model is singular † †

The table of model comparisons above indicates that the model with the complete random effects structure is singular. Removing the random slope of the highest-order interaction with zero variance, still leads to a singular model. Removing the next zero variance component leads to a model that we call the maximum identifiable model (here Model 2). This model is better than the model without random slopes as can be seen from the model comparison indices in that table.

2.3 Results from the maximum identified model

Figure 2. Estimated marginal means from the maximum identified model on response time data (Model 2). Individual participants' conditional modes are illustrated with smaller symbols and thin lines connecting the valid and invalid preview points. The preview effect, the difference between valid and invalid preview trials, depended on the training condition. In contrast to valid training (left side), there was no evidence for a preview effect with invalid training (right side, see also Models 2a and 2b below). Note that effect estimates were obtained for the first trial of the test phase. Error bars represent asymptotic confidence intervals.

Figure 2. Estimated marginal means from the maximum identified model on response time data (Model 2). Individual participants’ conditional modes are illustrated with smaller symbols and thin lines connecting the valid and invalid preview points. The preview effect, the difference between valid and invalid preview trials, depended on the training condition. In contrast to valid training (left side), there was no evidence for a preview effect with invalid training (right side, see also Models 2a and 2b below). Note that effect estimates were obtained for the first trial of the test phase. Error bars represent asymptotic confidence intervals.

Figure 3. Fixed effect coefficients of the maximum identified linear mixed model on response times (Model 2). Error bars represent 95% profile confidence intervals. Effect contrasts are given in brackets next to the names of main effects on the y-axis. In-Up: Inverted minus upright; Inv-Val: Invalid minus valid.

Figure 3. Fixed effect coefficients of the maximum identified linear mixed model on response times (Model 2). Error bars represent 95% profile confidence intervals. Effect contrasts are given in brackets next to the names of main effects on the y-axis. In-Up: Inverted minus upright; Inv-Val: Invalid minus valid.

Fixed effects of Model 2. Estimate, standard error, t-value, and lower/upper limit of 95% profile confidence intervals. This table contains the data plotted in Figure 3.
Parameter Estimate Std. Error t value 2.5 % 97.5 %
((Intercept)) -1.022 0.038 -27.213 -1.095 -0.948
Target Orientation (In-Up) 0.037 0.010 3.749 0.018 0.056
Preview (Inv-Val) 0.041 0.009 4.549 0.024 0.059
Training (Inv-Val) -0.138 0.075 -1.831 -0.285 0.009
Trial number -0.077 0.017 -4.388 -0.111 -0.042
Target Orientation x Preview -0.014 0.016 -0.841 -0.046 0.018
Target Orientation x Training 0.002 0.020 0.083 -0.037 0.040
Preview x Training -0.070 0.018 -3.848 -0.105 -0.034
Target Orientation x Trial number 0.013 0.007 1.866 -0.001 0.027
Preview x Trial number 0.003 0.007 0.425 -0.010 0.016
Training x Trial number 0.058 0.035 1.676 -0.010 0.127
Target Orientation x Preview x Training 0.010 0.033 0.300 -0.054 0.074
Target Orientation x Preview x Trial number 0.012 0.014 0.854 -0.015 0.038
Target Orientation x Training x Trial number -0.002 0.014 -0.122 -0.029 0.026
Preview x Training x Trial number 0.028 0.014 2.051 0.001 0.055
Target Orientation x Preview x Training x Trial number -0.017 0.027 -0.618 -0.070 0.037
Figure 4. Response times showed an interaction of Training x Preview x Trial Number, which suggested that training with only valid trials resulted in a larger preview effect than training with only invalid trials particularly in the beginning of the test phase. The preview effect then evolved in opposite directions for both training groups. Compared to the invalid training group, the preview effect in the valid training group declined. Dots represent a random sample of half of all individual data points. Trial number was standardized and centered on the first trial of the test phase.

Figure 4. Response times showed an interaction of Training x Preview x Trial Number, which suggested that training with only valid trials resulted in a larger preview effect than training with only invalid trials particularly in the beginning of the test phase. The preview effect then evolved in opposite directions for both training groups. Compared to the invalid training group, the preview effect in the valid training group declined. Dots represent a random sample of half of all individual data points. Trial number was standardized and centered on the first trial of the test phase.

Note, Figure 4 is zoomed-in at the y-axis.

2.4 Valid and invalid training groups analysed separately

Here we follow up the interaction Preview x Training x Trial Number to see whether the preview effects are significant within the training groups.

Supplementary Table S2. Fixed effects of Model 2a, the maximum identified model on response times of the valid training group. Estimate, standard error, t-value, and lower/upper limit of 95% profile confidence intervals.
Parameter Estimate Std. Error t value 2.5 % 97.5 %
((Intercept)) -0.953 0.056 -17.112 -1.065 -0.841
Target Orientation (In-Up) 0.036 0.013 2.799 0.011 0.061
Preview (Inv-Val) 0.076 0.015 5.154 0.047 0.105
Trial number -0.106 0.030 -3.525 -0.166 -0.045
Target Orientation x Preview -0.019 0.024 -0.800 -0.067 0.028
Target Orientation x Trial number 0.014 0.009 1.524 -0.004 0.032
Preview x Trial number -0.011 0.009 -1.187 -0.029 0.007
Target Orientation x Preview x Trial number 0.021 0.019 1.124 -0.016 0.057
Supplementary Table S3. Fixed effects of Model 2b, the maximum identified model on response times of the invalid training group. Estimate, standard error, t-value, and lower/upper limit of 95% profile confidence intervals.
Parameter Estimate Std. Error t value 2.5 % 97.5 %
((Intercept)) -1.091 0.051 -21.520 -1.193 -0.989
Target Orientation (In-Up) 0.038 0.015 2.563 0.009 0.067
Preview (Inv-Val) 0.007 0.011 0.607 -0.015 0.028
Trial number -0.047 0.019 -2.535 -0.084 -0.010
Target Orientation x Preview -0.007 0.022 -0.316 -0.051 0.036
Target Orientation x Trial number 0.012 0.011 1.053 -0.011 0.034
Preview x Trial number 0.017 0.010 1.676 -0.003 0.036
Target Orientation x Preview x Trial number 0.002 0.020 0.118 -0.037 0.041

2.5 Response times summary

The Preview x Training x Trial Number interaction is significant. Note that in the figure illustrating this interaction, the preview effect is the difference between dashed (invalid preview) and solid (valid preview) lines in the direction of the y-axis. If the training phase was valid, there is a preview effect in the beginning of the following test phase which decreases in the course of the test phase. If training phase was invalid, there is a smaller/no preview effect in the beginning of the following test phase which then, compared to the valid training condition, tends to increase. In other words, the influence of training equals across time.

Besides this interaction, there is a significant main effect of Target Orientation. This effect is in the expected direction known from previous research, i.e. faster responses with upright than with inverted targets. The direction of the effect can be seen from the contrasts of the Target Orientation factor and the value of the effect estimate. The contrast is In-Up, meaning inverted minus upright. That means the effect estimate is calculated by subtracing upright target trials from inverted target trials. That means positive values indicate larger dependent variable values for inverted than for upright targets. The dependent variable transformation of -1 / RT before model fitting ensured that larger values still mean slower responses (i.e. maintain the direction of the effect). Thus, given a positive value for the Target Orientation effect estimate () and confidence intervals excluding zero, we can conclude that responses were significanly faster with upright than with inverted targets.

3 Error rate / proportion correct

For details on the model fitting and model comparison approach, see the corresponding Rmd source code file.

3.1 Number of trials

Trials in the error rate analysis: 15765 , i.e. 87.8175134% of a total number of 17952 trials.

3.2 Model comparisons

Supplementary Table S4. Error rate model comparisons to determine the random effects structure. Each column presents the parameters (random and fixed) of a model. Models in adjacent columns are compared to each other by likelihood ratios tests. Test results (χ2, degrees of freedom, and p value) for a model pair are printed for the right model of each pair in the last three rows. Observations - the number of single trials for the model; AICc - Akaike’s Information Criterion corrected for small sample sizes; Df - model degrees of freedom; χ2 - statistic for the likelihood ratio test, for each model the difference in deviance compared to the model to the left; χ2 Df - degrees of freedom for the likelihood ratio test, for each model the difference in Df compared to the model to the left; p - the p-value for the likelihood ratio test.
Dependent variable:
Task error (log odds)
(5) (6) (7) (8)
Random effects variances
Participant
(Intercept) 0.344 0.340 0.340 0.340
Target Orientation (In-Up) 0.134 0.134 0.134
Preview (Inv-Val) 0.027 0.027 0.027
Trial number 0.032 0.032 0.032
Target Orientation x Trial number 0.043 0.043 0.043
Target Orientation x Preview x Trial number 0.136 0.136 0.136
Preview x Trial number 0 0
Target Orientation x Preview 0
Residual Variance 1 1 1 1
Fixed effects
Target Orientation (In-Up) 0.655 0.676 0.676 0.676
(0.085) (0.107) (0.107) (0.107)
t = 7.725 t = 6.331 t = 6.331 t = 6.331
Preview (Inv-Val) -0.107 -0.101 -0.101 -0.101
(0.085) (0.090) (0.090) (0.090)
t = -1.265 t = -1.125 t = -1.125 t = -1.125
Training (Inv-Val) 0.082 0.070 0.070 0.070
(0.216) (0.216) (0.216) (0.216)
t = 0.381 t = 0.326 t = 0.326 t = 0.326
Trial number 0.020 -0.003 -0.003 -0.003
(0.042) (0.054) (0.054) (0.054)
t = 0.463 t = -0.048 t = -0.048 t = -0.048
Target Orientation x Preview 0.028 0.027 0.027 0.027
(0.169) (0.170) (0.170) (0.170)
t = 0.166 t = 0.158 t = 0.158 t = 0.158
Target Orientation x Training -0.045 -0.009 -0.009 -0.009
(0.170) (0.213) (0.213) (0.213)
t = -0.265 t = -0.043 t = -0.043 t = -0.043
Preview x Training -0.156 -0.160 -0.160 -0.160
(0.169) (0.179) (0.179) (0.179)
t = -0.921 t = -0.892 t = -0.892 t = -0.892
Target Orientation x Trial number -0.199 -0.177 -0.177 -0.177
(0.084) (0.093) (0.093) (0.093)
t = -2.363 t = -1.893 t = -1.893 t = -1.893
Preview x Trial number -0.023 -0.031 -0.031 -0.031
(0.084) (0.085) (0.085) (0.085)
t = -0.267 t = -0.370 t = -0.370 t = -0.370
Training x Trial number -0.115 -0.111 -0.111 -0.111
(0.084) (0.106) (0.106) (0.106)
t = -1.363 t = -1.053 t = -1.053 t = -1.053
Target Orientation x Preview x Training -0.071 -0.068 -0.068 -0.068
(0.339) (0.340) (0.340) (0.340)
t = -0.210 t = -0.200 t = -0.200 t = -0.200
Target Orientation x Preview x Trial number -0.057 -0.071 -0.071 -0.071
(0.169) (0.182) (0.182) (0.182)
t = -0.338 t = -0.391 t = -0.391 t = -0.391
Target Orientation x Training x Trial number 0.006 0.033 0.033 0.033
(0.169) (0.186) (0.186) (0.186)
t = 0.034 t = 0.178 t = 0.178 t = 0.178
Preview x Training x Trial number 0.027 0.022 0.022 0.022
(0.169) (0.170) (0.170) (0.170)
t = 0.160 t = 0.127 t = 0.127 t = 0.127
Target Orientation x Preview x Training x Trial number 0.088 0.105 0.105 0.105
(0.337) (0.363) (0.363) (0.363)
t = 0.261 t = 0.288 t = 0.288 t = 0.288
Grand mean -1.538 -1.543 -1.543 -1.543
(0.108) (0.108) (0.108) (0.108)
t = -14.216 t = -14.257 t = -14.257 t = -14.257
Observations 15,765 15,765 15,765 15,765
AICc 14817.779 14775.659 14777.665 14779.671
Log Likelihood -7391.87 -7365.797 -7365.797 -7365.797
Deviance 14783.741 14731.595 14731.595 14731.595
Df 17 22 23 24
χ2 52.146 0 0
χ2 Df 5 1 1
p < .001 1.000 1.000
Model is singular † †

The table of model comparisons above indicates that the model with the complete random effects structure is singular. Removing the random slope of the less interesting component with zero variance, still leads to a singular model. Removing the next zero variance component leads to the maximum identifiable model (Model 6). This model is better than the model without random slopes as can be seen from the model comparison indices in the table above.

3.3 Results from the maximum identified model

Figure 5. Fixed effect coefficients of the maximum identified generalized linear model on task errors (Model 6). Error bars represent 95% profile confidence intervals. Effect contrasts are given in brackets next to the names of main effects on the y-axis. In-Up: Inverted minus upright; Inv-Val: Invalid minus valid.

Figure 5. Fixed effect coefficients of the maximum identified generalized linear model on task errors (Model 6). Error bars represent 95% profile confidence intervals. Effect contrasts are given in brackets next to the names of main effects on the y-axis. In-Up: Inverted minus upright; Inv-Val: Invalid minus valid.

Fixed effects of Model 6. Estimate and lower/upper limit of 95% profile confidence intervals. This table contains the data plotted in Figure 5.
Parameter Estimate 2.5 % 97.5 %
((Intercept)) -1.543 -1.762 -1.328
Target Orientation (In-Up) 0.676 0.467 0.890
Preview (Inv-Val) -0.101 -0.278 0.076
Training (Inv-Val) 0.070 -0.363 0.503
Trial number -0.003 -0.114 0.102
Target Orientation x Preview 0.027 -0.307 0.361
Target Orientation x Training -0.009 -0.429 0.416
Preview x Training -0.160 -0.513 0.193
Target Orientation x Trial number -0.177 -0.359 0.012
Preview x Trial number -0.031 -0.198 0.135
Training x Trial number -0.111 -0.325 0.102
Target Orientation x Preview x Training -0.068 -0.735 0.599
Target Orientation x Preview x Trial number -0.071 -0.431 0.285
Target Orientation x Training x Trial number 0.033 -0.331 0.405
Preview x Training x Trial number 0.022 -0.312 0.355
Target Orientation x Preview x Training x Trial number 0.105 -0.610 0.820
Figure S1. The Target Orientation x Trial Number interaction. This interaction is strictly speaking not significant and anyway theoretically not relevant.

Figure S1. The Target Orientation x Trial Number interaction. This interaction is strictly speaking not significant and anyway theoretically not relevant.

Figure S2. Proportion of errors in Training and Preview conditions.

Figure S2. Proportion of errors in Training and Preview conditions.

3.4 Error rates summary

Clearly less task error with upright than with inverted targets. In addition, there is a borderline significant Target Orientation x Trial Number interaction in the direction of a decreasing target orientation effect (inverted minus upright) across the test phase (Figure S1). However, strickly speaking, this effect is not significant and it is theoretically not relevant, so we do not mention it in the paper.

4 Results summary

Training influenced the preview effect in response times. In the beginning of the test phase, there was a clear preview effect if participants had trained with only valid trials. However, after invalid training there was no evidence for a preview effect. Moreover, this change in the preview effect equalled during the test phase.

In addition but theoretically not relevant, target face orientation, affected performance leading to slower responses and more errors when target faces were inverted compared to when they were upright.

5 Training phase data

The experiment consisted of a training and a test phase. For the main manuscript, we only analysed the data of the test phase, because only that phase was relevant. Here, we also analyse the training phase data to get an idea of why the valid training group gave slower responses than the invalid training group at the start of the test phase.

5.1 Response time

5.1.1 Model comparisons

Supplementary Table S5. Training phase response time model comparisons to determine the random effects structure. Each column presents the parameters (random and fixed) of a model. Models in adjacent columns are compared to each other by likelihood ratios tests. Test results (χ2, degrees of freedom, and p value) for a model pair are printed for the right model of each pair in the last three rows. Observations - the number of single trials for the model; AICc - Akaike’s Information Criterion corrected for small sample sizes; Df - model degrees of freedom; χ2 - statistic for the likelihood ratio test, for each model the difference in deviance compared to the model to the left; χ2 Df - degrees of freedom for the likelihood ratio test, for each model the difference in Df compared to the model to the left; p - the p-value for the likelihood ratio test.
Dependent variable:
-1 / Response time [sec]
Fitting method:
REML REML REML REML
(9) (10) (11) (12)
Random effects variances
Participant
(Intercept) 0.047 0.045 0.045r 0.045
Target Orientation (In-Up) 0.001r 0.002r 0.002
Trial number 0.008 0.008r 0.008
Target Orientation x Trial number 0.001r 0.002r 0.002
Residual Variance 0.033 0.031 0.031 0.031
Fixed effects
Target Orientation (In-Up) 0.022 0.027 0.027 0.027
(0.007) (0.009) (0.010) (0.010)
t = 3.257 t = 3.173 t = 2.649 t = 2.599
Training (Inv-Val) 0.052 0.052 0.052 0.051
(0.074) (0.072) (0.072) (0.072)
t = 0.706 t = 0.721 t = 0.718 t = 0.715
Trial number -0.068 -0.065 -0.065 -0.065
(0.003) (0.015) (0.016) (0.016)
t = -20.315 t = -4.202 t = -4.184 t = -4.167
Target Orientation x Training -0.029 -0.022 -0.023 -0.023
(0.013) (0.017) (0.020) (0.021)
t = -2.127 t = -1.277 t = -1.124 t = -1.122
Target Orientation x Trial number 0.007 0.001 0.001 0.001
(0.007) (0.008) (0.009) (0.010)
t = 1.033 t = 0.149 t = 0.138 t = 0.148
Training x Trial number -0.020 -0.020 -0.019 -0.019
(0.007) (0.031) (0.031) (0.031)
t = -2.970 t = -0.635 t = -0.625 t = -0.617
Target Orientation x Training x Trial number 0.041 0.034 0.035 0.035
(0.013) (0.016) (0.019) (0.019)
t = 3.078 t = 2.156 t = 1.849 t = 1.854
Grand mean -0.953 -0.954 -0.954 -0.954
(0.037) (0.036) (0.036) (0.036)
t = -25.873 t = -26.559 t = -26.576 t = -26.497
Observations 12,334 12,334 12,334 12,334
AICc -6660.63 -7298.309 -7300.719 -7296.973
Log Likelihood 3340.324 3662.169 3664.377 3667.517
Deviance -6680.648 -7324.338 -7328.753 -7335.034
Df 10 13 14 19
χ2 643.691 4.415 6.281
χ2 Df 3 1 5
p < .001 .036 .280

5.1.2 Results from the maximum identified model

Figure S3. Training phase data. Fixed effect coefficients of the best maximum identified linear mixed model on response times (Model 11). Effect contrasts are given in brackets next to the names of main effects on the y-axis. In-Up: Inverted minus upright; Inv-Val: Invalid minus valid.

Figure S3. Training phase data. Fixed effect coefficients of the best maximum identified linear mixed model on response times (Model 11). Effect contrasts are given in brackets next to the names of main effects on the y-axis. In-Up: Inverted minus upright; Inv-Val: Invalid minus valid.

Fixed effects of Model 11. Estimate, standard error, t-value, and lower/upper limit of 95% profile confidence intervals. This table contains the data plotted in Figure S3.
Parameter Estimate Std. Error t value 2.5 % 97.5 %
((Intercept)) -0.954 0.036 -26.576 -1.025 -0.884
Target Orientation (In-Up) 0.027 0.010 2.649 0.007 0.047
Training (Inv-Val) 0.052 0.072 0.718 -0.089 0.192
Trial number -0.065 0.016 -4.184 -0.095 -0.035
Target Orientation x Training -0.023 0.020 -1.124 -0.063 0.017
Target Orientation x Trial number 0.001 0.009 0.138 -0.017 0.020
Training x Trial number -0.019 0.031 -0.625 -0.080 0.041
Target Orientation x Training x Trial number 0.035 0.019 1.849 -0.002 0.072
Figure S4. Left panel: Response time across trials in the training phase consisting one group of participants of only valid trials and for another group of participants of only invalid trials (Model 11). Right panel: Response time across trials in the test phase which consisted of 50% valid and invalid trials for all participants (right panel, Model 2, identical to Figure 4). Dots represent a random sample of half of all individual data points. Note that trial numbers were standardized and centered on the first trials within each phase.

Figure S4. Left panel: Response time across trials in the training phase consisting one group of participants of only valid trials and for another group of participants of only invalid trials (Model 11). Right panel: Response time across trials in the test phase which consisted of 50% valid and invalid trials for all participants (right panel, Model 2, identical to Figure 4). Dots represent a random sample of half of all individual data points. Note that trial numbers were standardized and centered on the first trials within each phase.

Note, Figure S4 is zoomed-in at the y-axis.

5.1.3 Training phase response time summary

As can be seen from the model coefficients in Figure S3 and the response time data across trials in Figure S4, participants in the invalid training condition showed numerically, though not significantly, slower response times than the valid training participants in the beginning of the training phase. This difference numerically declined throught the training phase. At the start of the test phase, both groups of participants were about equally fast. Interestingly, at the beginning of the test phase, the invalid training group still showed about the same response time whereas the valid training group showed significantly slower responses.

5.2 Error rate

5.2.1 Model comparisons

Supplementary Table S6. Training phase error rate model comparisons to determine the random effects structure. Each column presents the parameters (random and fixed) of a model. Models in adjacent columns are compared to each other by likelihood ratios tests. Test results (χ2, degrees of freedom, and p value) for a model pair are printed for the right model of each pair in the last three rows. Observations - the number of single trials for the model; AICc - Akaike’s Information Criterion corrected for small sample sizes; Df - model degrees of freedom; χ2 - statistic for the likelihood ratio test, for each model the difference in deviance compared to the model to the left; χ2 Df - degrees of freedom for the likelihood ratio test, for each model the difference in Df compared to the model to the left; p - the p-value for the likelihood ratio test.
Dependent variable:
Task error (log odds)
(13) (14) (15)
Random effects variances
Participant
(Intercept) 0.316 0.539 0.539
te1.mm[, “TargOrientIn-Up”] 0.349 0.349
te1.mm[, “TrialNum.1z”] 0.269 0.269
te1.mm[, “TargOrientIn-Up:TrialNum.1z”] 0
Residual Variance 1 1 1
Fixed effects
Target Orientation (In-Up) 0.432 0.492 0.492
(0.081) (0.131) (0.131)
t = 5.304 t = 3.745 t = 3.745
Training (Inv-Val) 0.183 0.158 0.158
(0.207) (0.263) (0.263)
t = 0.884 t = 0.600 t = 0.600
Trial number -0.178 -0.200 -0.200
(0.042) (0.098) (0.098)
t = -4.272 t = -2.028 t = -2.028
Target Orientation x Training -0.116 -0.118 -0.118
(0.163) (0.262) (0.262)
t = -0.712 t = -0.449 t = -0.449
Target Orientation x Trial number -0.107 -0.058 -0.058
(0.083) (0.086) (0.086)
t = -1.288 t = -0.675 t = -0.675
Training x Trial number -0.108 -0.121 -0.121
(0.083) (0.197) (0.197)
t = -1.296 t = -0.615 t = -0.615
Target Orientation x Training x Trial number -0.073 0.016 0.016
(0.166) (0.171) (0.171)
t = -0.439 t = 0.094 t = 0.094
Grand mean -1.320 -1.352 -1.352
(0.104) (0.132) (0.132)
t = -12.736 t = -10.261 t = -10.261
Observations 15,485 15,485 15,485
AICc 14935.761 14720.796 14722.799
Log Likelihood -7458.875 -7349.389 -7349.389
Deviance 14917.75 14698.779 14698.779
Df 9 11 12
χ2 218.971 0
χ2 Df 2 1
p < .001 1.000
Model is singular †

5.2.2 Results from the maximum identified model

Figure S5. Training phase data. Fixed effect coefficients of the best maximum identified linear mixed model on error rates (Model 14). Effect contrasts are given in brackets next to the names of main effects on the y-axis. In-Up: Inverted minus upright; Inv-Val: Invalid minus valid.

Figure S5. Training phase data. Fixed effect coefficients of the best maximum identified linear mixed model on error rates (Model 14). Effect contrasts are given in brackets next to the names of main effects on the y-axis. In-Up: Inverted minus upright; Inv-Val: Invalid minus valid.

Fixed effects of Model 14. Estimate and lower/upper limit of 95% profile confidence intervals. This table contains the data plotted in Figure S5.
Parameter Estimate 2.5 % 97.5 %
((Intercept)) -1.352 -1.620 -1.089
Target Orientation (In-Up) 0.492 0.233 0.755
Training (Inv-Val) 0.158 -0.373 0.688
Trial number -0.200 -0.398 -0.002
Target Orientation x Training -0.118 -0.639 0.405
Target Orientation x Trial number -0.058 -0.226 0.110
Training x Trial number -0.121 -0.518 0.274
Target Orientation x Training x Trial number 0.016 -0.320 0.352

5.2.3 Training phase error rates summary

In the training phase error rates, there is an effect of Target Orientation as we know it (less errors with upright than with inverted targets) and an effect of Trial Number with a negative coefficient meaning that errors decrease across the training phase.

6 Training phase summary and conclusions

The response time data of the training phase provides some evidence that invalid trials were actually surprising. The valid training group saw the invalid trails for the first time in the beginning of the test phase whereas the invalid training group was already used to the invalid trials. At the end of the training phase, both groups were about equally fast, but at the start of the test phase, the valid training group showed significantly slower responses. Moreover, for the valid training group, responses were particularly slowed down for invalid trials (cf. Figure S4 above). This pattern suggests that the invalid trials were indeed unexpected for the valid training group. In contrast, valid trials did not seem to be a big surprise for the invalid training group, supposedly because of the everyday life experience that objects usually do not change during saccades. Credit to an anonymous reviewer for this insight. Error rates did not show any effects worth mentioning; they were simply lower for upright than for inverted targets and they decreased across the training phase.

7 Session information

R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Catalina 10.15.6

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] C

attached base packages:
[1] grid      stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] knitr_1.24        citr_0.3.2        stargazer_5.2.2   boot_1.3-23       emmeans_1.4.5     lme4_1.1-21       Matrix_1.2-18     gtable_0.3.0     
 [9] ggplot2_3.2.1     data.table_1.12.2

loaded via a namespace (and not attached):
 [1] gtools_3.8.1     tidyselect_1.1.0 xfun_0.9         reshape2_1.4.3   purrr_0.3.2      splines_3.6.1    lattice_0.20-38  colorspace_1.4-1
 [9] vctrs_0.3.4      generics_0.0.2   miniUI_0.1.1.1   htmltools_0.3.6  yaml_2.2.0       rlang_0.4.7      pillar_1.4.2     nloptr_1.2.1    
[17] later_0.8.0      glue_1.4.2       withr_2.1.2      plyr_1.8.4       lifecycle_0.2.0  stringr_1.4.0    munsell_0.5.0    mvtnorm_1.0-11  
[25] coda_0.19-3      evaluate_0.14    labeling_0.3     httpuv_1.5.1     highr_0.8        Rcpp_1.0.2       xtable_1.8-4     promises_1.0.1  
[33] scales_1.0.0     mime_0.7         digest_0.6.20    stringi_1.4.3    dplyr_1.0.2      shiny_1.3.2      tools_3.6.1      magrittr_1.5    
[41] lazyeval_0.2.2   tibble_2.1.3     crayon_1.3.4     pkgconfig_2.0.2  MASS_7.3-51.4    estimability_1.3 assertthat_0.2.1 minqa_1.2.4     
[49] rmarkdown_1.15   rstudioapi_0.10  R6_2.4.0         nlme_3.1-140     compiler_3.6.1